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Abstract 

This article describes the evaluation of an online professional development program funded by the State of 
Colorado to address the need for highly qualified science teachers in high need and/or rural school districts. 
Recruitment and the retention of highly qualified educators in high need and/or rural school districts is a critical 
factor affecting the education and the possible career trajectory of students enrolled in science, technology, 
engineering and math (STEM) related courses. The program describes the participants experiences with the 
online format, the educational courses offered, and the overall effect of the program. The results of the 
evaluation provided positive results for participants experience and valuable information regarding 
improvements to such programs. 

Keywords: 

1. Introduction 

The National Academies' Rising Above the Gathering Storm (2007) report calls for action to support U.S. 
science, technology, engineering and mathematics (STEM) competitiveness for the coming century, and 
specifically to “focus on actions in K-12 education.” Like most places in the country, our state is experiencing a 
shortage of highly qualified teachers in the STEM areas. These shortages are compounded in high poverty and 
rural areas, where fewer teachers generally seek employment, are not able to conveniently complete 
requirements to be considered highly qualified in all subjects they teach, and are unable to access content 
specific professional development. Researchers claim professional development opportunities for teachers in 
rural schools contributes to the achievement and success of students (Beesley, 2011), Yet, supporting teachers 
with content specific professional development in low population areas is complex, and providing opportunities 
for teachers in more remote locations, such as mountain communities, is even more difficult. Single professional 
development offerings have limited value, and research supports multiple session instructional courses as an 
effective method for providing ongoing professional development (Gaumer Erickson, Noonan, & McCall, 2012). 
However, ongoing offerings involve many logistical factors and concerns that must be addressed prior to starting 
a professional development program and distance education provides an added layer of complexity. Elements of 
concern include the management, organization, communication, content delivery, time requirements, cost, and 
other individual participant concerns (Engstrom, Santo & Yost, 2008). Programs should be based on ongoing 
activity based projects, which are relationship-based, occur in the teachers’ environment and encourages the 
sharing of ideas with peers (Kenny, Seen & Purser, 2008). 

Online options for professional development have the potential to solve some of the issues related with 
rural schools and professional development. The online environment makes it possible to access courses from 
distant organizations, contribute/communicate with a community of peers, can be cost effective, and may have 
flexibility regarding the time requirements of the program (Beesley, 2011). 

In order to address these challenges, Colorado State University’s School of Teacher Education and 
Principal Preparation (STEPP), instituted a project, funded by an Improving Teacher Quality grant through the 
Colorado Department of Higher Education (CDHE), to provide high quality professional development for 
teachers in high need schools. Colorado State University (CSU) was already uniquely poised to support the 
needs of several high schools in rural and urban areas throughout Colorado with a state wide partnership of high- 
need schools (CSU Alliance School Program) that provided an immediate connection to identify and address 
teacher needs. 

The goal of this newly formed partnership with the Alliance School Network was to increase science 
teachers’ knowledge and qualifications in 10 high needs rural/small schools and urban high schools throughout 
Colorado. Addressing both the challenges of high need schools and the critical teaching area, the project also 
provided content specific coursework for other licensed teachers to become highly qualified to teach science. 
Participants completed three (pilot) online courses, developed by CSU’s College of Natural Sciences, and 
implemented course material into their own K-12 classes—evaluating the impact on student achievement and 
providing feedback to professors to refine courses for future use. The courses focused on Biology (NSCI 650), 
Physics (NSCI 619), and Chemistry (NSCI 620) and were designed with CSU Alliance School input and support 
from CSU’s Institute for Teaching and Learning (TILT) to ensure online structure was aligned with project 
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objectives. STEPP faculty supervised the overall project and served as project coordinator for the collaboration, 
communication, organization, recruitment and documented project progress. 

The three pilot courses were an initial step in a five course online program for an Advanced Science 
Instruction Certificate (ASIC) contributing to a Master of Natural Science Education (MNSE) degree. 
Participants received funding for the tuition of the three selected online courses, textbook purchase and a 
classroom/lab material stipend. Each participant had a selected site faculty member that received funding to 
serve as an educational coach to provide observational and collaborative support during the grant period. BSCS, 
a non-profit science education research and development organization, was contracted to evaluate the project. 
Qualitative and quantitative approaches were utilized to examine the impact of the Improving Teacher Quality 
grant on teacher learning, teacher practice, and student learning. BSCS monitored and provided feedback on the 
quality of the online course content and instruction, the use technologies for course delivery, and engagement of 
teachers in sustainable professional learning communities. The unpublished manuscript of their evaluative results 
was provided at the conclusion of the project, and these results have been used for this article (BSCS, 2012). 

2. Materials and Methods 

2.1 Participants 

The initial grant proposal was designed to support 18 secondary science teachers. Due to the submission and 
funding timeline, 14 teachers were recruited for the initial course, NSCI 650, for the spring 2011 semester. 
Following the initial launch of the program, 5 additional teachers were selected to participate in the program. 
NSCI 650 was repeated in the summer of 2011 to facilitate the accommodation for the added participants. A total 
of 19 participants representing 12 different schools from around Colorado participated in some or all of the 
aspects of the program. 5 participants completed all of the required evaluative activities and 14 partially 
completed various elements of the evaluation. 

2.2 Evaluative Design 

As mentioned previously, the evaluation of the grant program incorporated both qualitative and quantitative data 
collection and analysis. The evaluation of the online courses as experienced by the participants was completed 
utilizing qualitative methods, and a combination of qualitative and quantitative methods were used to evaluate 
teacher learning and practice. 

2.2.7 Online course evaluation 

The online course evaluation focused on 3 themes: (a) the quality of course content and instruction by examining 
(i) laboratory exercises,(ii) lesson plans, and the (iii) content/pedagogy interface (b) use of technologies for 
course delivery (c) engagement of teachers in sustainable professional learning communities. 

Data related to the online courses were collected and analyzed from the following sources: 

• CSU course reports at the conclusion of the 4 sections of courses that consisted of Likert-type responses 

• Project-designed surveys administered at the conclusion of the courses consisting of Likert and open- 
ended responses. 

• Observational records of online discussions among students and professors for the courses 

• Notes from meetings with project leadership and course instructors. 

2.2.2 Teacher learning and practice 

The evaluation of the teacher learning and practice was completed utilizing selected assessments and employing 
the grant-funded instructional coaches as assessment proctors. Assessments were administered by the 
instructional coaches selected to work with each participant at their location. The responsibility of the coach was 
to conduct at least one in class observation per semester, engage in pre/post observation conferences, proctor the 
science content and pedagogical knowledge assessment, complete of surveys related to coaching experiences and 
support the participant on an as-needed basis during the grant time period. Teacher learning was evaluated 
using: 

• Pre/post assessment of science content and pedagogical knowledge using the Diagnostic Teacher 
Assessments in Mathematics and Science (DTAMS) (University of Louisville, 2004) during the spring 
2011 and spring 2012 semesters. 

• Self-assessment of content knowledge recorded in 3 post-semester surveys. 

Teacher practice was evaluated using: 

• Science Classroom Observation Protocol (SCOP) (RMC Research Corporation, 2010) scores reported 
by coaches from pre/post conferences and classroom observations of course content implementation for 
three semesters. 

• Participant responses on three end-of-course surveys about a) the impact of the coaching experience and 
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b) barriers to learning and/or implementing course content. 

• Instructional coaches’ responses to survey items related to their experiences. 

• Administrators’ responses to survey items related to the participants’ experiences. 

The DTAMS assessment consists of twenty multiple choice items and four open-response items in three 
content areas: Physical Science, Life Science, and Earth/Space Science. Items are categorized by knowledge type 
(Declarative, Inquiry, or Schematic). The open-response items are combined for a score in Pedagogy (knowledge 
representing strategic knowledge for teaching science). The evaluation of the DTAMS results were completed 
using a paired sample /-test to compare the pre and post test scores in the three content areas and the pedagogy 
measurement. Nineteen participants completed the DTAMS pre-test, and twelve the DTAMS post-test, resulting 
in twelve matched pairs for analysis. 

3. Results 

3.1 Laboratory exercises 

The participants provided positive responses related to the hands-on lab kits that accompanied course instruction. 
Lab kits were purchased with grant funding, assembled and sent to participants for use during each course. 
Participant comments from survey items included: 

• The labs were fantastic, when I understood what to do. Getting the lab supplies is the highlight of the 
class. As a visual learner, the YouTube videos are great. Maybe use this to help with the lab setup. 
(NSCI619) 

• The labs were extremely beneficial. Not only did they help me with understanding the material, it will 
help students in high school classrooms. The equipment is inexpensive, which is good for a school 
budget. (NSCI 620) 

• I liked the lab kits and projects. (NSCI 620) 

3.2 Lesson plans 

During the first course of the series, the CSU professor worked with advisors at TILT to develop a lesson plan 
format and peer-evaluation guide. After the initial offering, students were allowed to develop their own lesson 
plan format for NSCI 619. The project-designed survey included two items related to the lesson plan evaluation 
tool and lesson plan evaluation process; these items received the lowest ratings of the fourteen items. At the 
conclusion of NCSI 619 nine instructor prepared survey items related to evaluation of ‘lesson plan assignments 
and reviews’ received the lowest rating of 3.3/5. The CSU course survey items for ‘lesson plans done in groups’ 
and ‘group lesson plan’ were identified as assignments that contributed least to the experiences in the course. 
Student comments related to the lesson plan included: 

• “Lesson plans were not set up to be successful the first round as we did not have the expectations. 
When expectations were laid out, they were not completely relevant to a high school classroom. I would 
have rather had less management specific evaluation and more content relevant evaluation of my 
plan. ” (NSCI 650/ Spring) 

• “More flexibility in the lesson plans because it was difficult to fit all of the required criteria into one 
lesson that fit into the course I am teaching. ” (NSCI 650/Spring) 

• “The requirement to write a lesson plan was tedious. Some of the plans I will never use. It was time 
consuming. I would rather have a discussion question to respond to, it would have been more useful to 
hear how others are teaching certain content. ” (NSCI 620) 

• “I also would like to see the lesson plan requirement changed... or at least a sample lesson plan needs 
to be available. There were all levels of effort in the plans that I read, some short and cryptic, others 
long. Since they were worth 50 points, it would be helpful to know what a 50-point lesson plan looks 
like; especially since we are grading each other’s plans. ” (NSCI 620) 

3.3 Content/Pedagogy interface 

Course survey items related to the content/pedagogy interface received particular attention by participants 
interested in the application of the science course content within their classroom. Comments regarding this 
theme focused on the ‘fit’ of course content with middle and high school science standards and curricula, and the 
expected integration of pedagogical components with course content to facilitate instruction. The CSU course 
survey concentrated on the instructor’s delivery of course material, the project-designed survey focused more on 
the elements of the course that were related to the participants’ classroom teaching practice. Two items from the 
project-designed survey that received relatively low ratings included: Item 5: 1 was able to effectively implement 
some of the course content into my class (es) and item 14: Overall, the course content was relevant to my 
practice. On a 4-point scale these items received a rating of 2.78 and 2.87 for NSCI 619 and NSCI 620, a much 
lower score than other items included on the survey. Student comments included: 
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• “It was often difficult to incorporate such advanced content into a 9 th grade biology course and what 
we were learning fit into what we were doing in the class at that particular time. ” (NSCI 650) 

• “The content is pretty high level for 7 th graders and doesn ’t always relate to our current standards and 
outcomes. ” (NSCI 650) 

• “Tightly circumscribed Colorado Content Standards impaired the implementation of specific content in 
my classroom, but did not impair my own learning. ” (NSCI 620) 

• “It was very difficult to implement much of what I was learning in the course because they did not fit 
with the current unit I was teaching and were so far beyond what I would be teaching my students. ” 
(NSCI 620) 

• “My understanding was that this course was designed to improve teacher knowledge in the classroom, 
yet optics is not an area that is addressed in the Colorado State Standards. I would have benefited 
more by something I would more likely teach in my classroom, like say Forces, Newton’s laws, etc. ” 
(NSCI 619) 

• “To focus all on one topic I don’t think served the student population (teachers in the field!) Well...But 
give us a real “Physics for Science Educators” class that delivers what’s advertised. ” (NSCI 619) 

• ”1 do not feel this course was designed for educators. ” (NSCI 620) 

• “Include more ways to increase student engagement in the classroom. ” (NSCI 650) 

• “More of an integration of teaching practice in addition to the content. ” (NSCI 620) 

3.4 Use of Technologies for Course Delivery 

CSU professors and the Institute for Learning and Teaching collaborated to develop an online learning platform 
for integration of course content for each 15-week course. The online format allowed for a student access to 
course content, learning activities discussion forums, related websites, videos, software, study guides and 
interactive components. Two items on the CSU course survey specifically addressed the participants’ interaction 
with the online learning experience. Both items rated relatively high; a mean score of 4 on a 5-point scale (80%). 
The project designed survey included the statement “I felt comfortable using the online format.” The mean score 
for responses was a 3 (agree) on a 4-point scale (75%). Participant comments included: 

• “I started out very uncomfortable with the online process but grew to understand it much better by the 
end. Because of my learning cur\’e with online classes I found myself behind from the beginning and 
struggled to get caught up. But I’m ready for next fall now. ” (NSCI 650/Spring) 

• “I really liked having instructor’s notes, and an assignment overview. That was SUPER helpful to 
ensure I had everything completed. The online format at CSU seems to be improving!” (NSCI 619) 

• “Never underestimate the power of a good 1 minute video to explain the math for certain scenarios. A 
video helps replace what we are missing in a lecture hall or during office hours. ” 

• “The online lectures should not be streamed. They MUST be provided in a downloadable format so they 
can be viewed, stopped, rewound, or reviewed without an active internet connection. ” (NSCI 620) 

The highest score for course components in NSCI 619 (4.7/5), was obtained for videos of demonstrations by the 
instructor and labs (with lab kit materials) in the instructor-prepared survey items. 

3.5 Engagement of Teachers in Sustainable Professional Learning Communities 

CSU instructors provided opportunities for participants to interact online through course discussion groups, chat 
rooms, and peer editing of lesson plans. Three items on the CSU course survey asked participants to provide 
feedback regarding the interactive community, items listed on the survey included: 

• How well did the instructor create an atmosphere that was respectful of student opinions? 

• How well did the instructor create an atmosphere that was respectful of student ideas and differences? 

• How was the instructor’s ability to encourage and facilitate interactive discussions? 

Across the series of courses, participants collectively scored the respectful atmosphere highest and opportunities 
for interaction lowest of the three items. Survey results indicated an increase between the first (spring) and the 
second (summer) sessions of NSCI 650. The increase may be attributable to participants’ and instructors’ 
comfort level with the online format and communication opportunities. Throughout the series of courses, 
additional or modified discussion forums were present. The mean messages per participant (including instructor) 
increased from the initial course (NSCI 650/Spring 2011) of 16.4 messages to the final course (NSCI 619/Spring 
2012) of 41 messages per participant. 

The project-designed survey included items related to the online professional learning communities and the 
interactive element of the courses, student comments included: 

• “The PLC aspect was not great; participants were pretty quiet, little discussion. ” (NSCI 650/Spring) 

• “I am not sold on online classes. I would like there to be more of an opportunity to work collaboratively 
on some projects. I also did not think the format added to a PLC, but perhaps that was just me. ” (NSCI 
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620) 

• “The most valuable aspect of the course was contact with other teachers in the state. ” (NSCI 620) 

• “The aspect that was most valuable to me was the interaction between myself and other science 
teachers around the state. ” (NSCI 620) 

• “I would like to have ongoing support of other students for the questions that come up. I am the only 
science teacher in my district and I don’t have anyone to go to with questions. ” (NSCI 620) 

• “I like reviewing lesson plans and sharing ideas, but working with another person was downright 
stressful and frustrating. ” (NSCI 619) 

• “Wasted my time trying to connect with other students. ” (NSCI 619) 

• “Maybe have a space where we can have a discussion with the instructor only. Sometimes my questions 
seem stupid and I don't want the whole class to see my level of ignorance. ” (NSCI 619) 

3.6 DTAMS content knowledge 

The DTAMS assessment for content knowledge addressed participant knowledge in earth science, life science, 
and physical science. Each specific content area was analyzed with a paired sample f-test to exam the mean 
difference between the pre-test and post-test assessment results. Physical science (N = 9) assessment results 
found no statistically significant difference between the pre-test (M = 17.44) and post-test (M = 18.56) 
assessment scores. Life science (N = 11) assessment results found no statistically significant difference between 
the pre-test ( M = 21.09) and post-test (M = 21.64) assessment scores. Earth science (N = 12) assessment results 
found no statistically significant difference between the pre-test (M = 18.17) and post-test (M = 21.00) 
assessment scores. Results indicate that participants’ content knowledge in each specific area increased as a 
result of participation, with the largest increase in earth science and the smallest increase in life science. None of 
the increases in content knowledge reached a statistically significant level, in part due to the small sample size 
that completed the corresponding pre- and post-test. The results suggest that engagement in the online natural 
sciences courses was beneficial and may have contributed to an increase in content knowledge. 

3.7 DTAMS pedagogical knowledge 

The DTAMS assessment for pedagogical knowledge addressed participant pedagogical knowledge in earth 
science, life science, and physical science. Each specific content area was analyzed with a paired sample Mest to 
exam the mean difference between the pre-test and post-test assessment results. Physical science (N = 9) 
assessment results indicated a statistically significant difference between the pre-test (M = 2.44) and post-test (M 
= 3.89) assessment scores t (8) = 3.250, p < .05. Life sciences (N = 11) assessment results found no statistically 
significant difference between the pre-test (M = 5.82) and post-test (M = 5.82) assessment scores. Earth science 
(N = 12) assessment results found not statistically significant difference between the pre-test (M = 5.76) and 
post-test (M = 6.00) assessment scores. The online natural sciences courses did not explicitly include 
pedagogical instruction. Results of the life sciences and earth sciences suggest confirmation of this fact, with 
none or minimal increases in mean pedagogical knowledge between the pre- and post-test. Pre- and post-test 
mean for the physical science increased and was statistically significant. However, extreme caution should be 
used before drawing any conclusions of the result. The sample size was incredible low and the pre- and post-test 
mean was dramatically different from the mean of the life science and earth science for pedagogical knowledge. 

3.8 Survey results on teacher learning 
3.8.1 Project-Designed Survey 

The project-designed survey asked participants’ their impressions of the natural sciences courses, the coaching 
experience and classroom implementation of course content. Surveys were administered to participants at the 
conclusion of the spring 2011, fall 2011 and spring 2012 semester online courses. The survey consisted of items 
reported on a 4-point Likert scale from 1-strongly disagree to 4-strongly agree. The mean rating for each course 
was 3.25/4 = 81% (NSCI 650), 3.22/4 = 80.5% (NSCI 620), and 3.67/4 = 92% (NSCI 619). The open-ended 
items included on the survey asked participants to provide feedback on the barriers to learning courses content, 
comments included: 

• “Some reading assignments were huge and some assigned problems were very difficult. I appreciate the 
challenge, but it was difficult to grade my own papers and also get my online homework done. ” (NSCI 
650) 

• “I haven’t had chemistry class since my undergraduate education 30 years ago. Being reintroduced to 
several of the topics in chemistry helped me realize what I do and do not know well and what I need to 
work on if I were ever to teach chemistry at the high school level. ” (NSCI 620) 

• “The majority of the course content was extremely too difficult for 7 ,h grade students. That was a huge 
barrier for me, but I am also aware that the master’s program was intended for high school teachers. ” 
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(NSCI 620) 

• “I need better math skills. I want better math skills, cam we have a class in teaching math!" (NSCI 
620) 

• “The greatest barrier I encountered was the fact that the course and its content didn’t have a big 
influence on my curriculum. Again, I believe there is a real big disconnect between higher and lower 
education and the expectations of a classroom teacher. ” (NSCI 619) 

3.8.2 CSU course surveys 

At the conclusion of each course, participants completed a standard survey utilized by University instructors’ 
and provide feedback regarding the teaching of the course. Items on the survey include; instructor ability, 
difficultly of course, reading assignments, other course assignments, class activities, etc. Items were rated on a 5- 
point Likert scale from 1-poor to 5-excellent. Survey items related to the contribution of class sessions, reading 
assignments, other course assignments, and learning resources to increase understanding of the subject were 
rated between 3 and 4.25 by participants. Items related to the intellectual challenge of each course were rated 
4.67, 4.83, 3.73 and 4.58. Participants rated relevance and usefulness of homework and reading assignments a 
4.33/5 for the course NSCI 650 Pollution and Environmental Biology for Educators. Use of laboratory kits and 
exercises was rated the lowest with 3.17/5 for the course. Responses to open-ended items for the course NSCI 
619, Physics for Science Educators asked for feedback related to activities or assignments that contributed most 
to students’ experience and learning included: 

• “The textbook...I’ve never really understood relativity, so the way it was presented in the text was very 
helpful. The homework questions were also challenging and actually fun to do; I felt like I had a better 
understanding of the content when finished. ” 

• "I felt like I learned the most from the lab activities and enjoyed them for the most part. ” 

• / enjoyed the instructor’s notes and the lab assignments. Having to take photos or describe the lab held 
me accountable to doing all the activities, and I learned a lot from my experiments. Many of them I can 
incorporate next fall into my classroom. ” 

• “The class was a bear but I did learn a ton. I love the labs! Lots of lab ideas and lab supplies, this 
great! ” 

3.9 Teacher Practice 

Instructional coaches were recruited from the school districts of the participants and served as mentors during the 
duration of the program. Coaches conducted in-class observations of the participants’ implementation and 
utilization of course information within their particular secondary science classroom. The Science Classroom 
Observation Protocol (SCOP) provided a structure for coaches to follow for the pre-conference, in-class 
observation, and post-conference activities. The SCOP instrument consisted of pre- and post-conference 
questions, and a detailed scoring rubric focusing on 13 items related to learning objectives, developing 
understanding, sense-making and classroom culture. Each item was scored on a scale of 0 to 6 points, with a 
grand total of 78 points. Coaches would utilize these documents once a semester when performing a formal in- 
class observation of their designated participant. 

Due to issues related to delayed recruitment of participants, scheduling of courses during summer, and 
delayed submission of SCOP documents; a total of five participants’ scores were analyzed and are documented 
in this article. Two participant SCOP scores included all three semester observations, and three included two 
semester observations scores. Results for the SCOP scores of the five participants were: 

• Participant 1: SCOP score increased from a score of 44 to 67 between the first and second semester, and 
increased from a 67 to 68 between the second and third semester. 

• Participant 2: SCOP scores varied between 47 for the first semester, 46 for the second, and 48 for the 
third semester. 

• Participant 3: SCOP score increased from 61 to 70 between the first and second semester, the 
participant did not complete an assessment for the third semester. 

• Participant 4: SCOP scores varied between a 73 in the first semester and a 74 for the second semester, 
the participant did not complete an assessment for the third semester. 

Participant 5: SCOP scores decreased from 43 in the first semester to 36 in the second semester, the participant 
did not complete an assessment for the third semester. 

3.9.1 Coaching 

The project-designed survey provided a more personalized perspective on the impact of the CSU TQP grant with 
the participants’ personal experiences with coaching and the transfer of course content to classroom instruction. 
The survey section that focused on coaching indicated participant satisfaction with the process, comments 
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included: 

• “The in-school coaching experience that I found most valuable was the feedback from my actual 
observations from my coach. I value their opinions in education and look forward to improving my 
teaching strategies with their help. ” (Spring 2012) 

• “It was great to get feedback so that I would have another perspective to reflect on. ” (Spring 2012) 

• “More interaction with my coach because I value her opinion and expertise in teaching. ” (Spring 2011) 

• “The frequency of the coaching visits because there did not seem to be very many. ” (Fall 2011) 

• “I wish my coach had a better and more in-depth knowledge of science to help me implement better 
strategies for science content. ” (Spring 2012) 

• “The part of the in-school coaching experience I would like most to change is to have various people 
make observations, especially someone who has more knowledge and a background in science. ” (Fall 
2011) 

The coaches’ surveyed responses indicated their agreement as moderate to high for: 

• Improvement of their personal coaching skills 

• Enhancement of participants’ science teaching skills 

• Enhancement of participants’ content knowledge 

• Enhancement of participants’ pedagogical content knowledge 

• The benefit of the TQP program for the school’s science program 

• Participants’ increased interest and achievement in classes of TQP teachers’ classrooms 

The coaches also indicated agreement of high to very high for the statement, “Implementing the SCOP was an 
educational experience for me as a coach.” Administrators indicated agreement of high to very high for the 
statements, “the teachers effectively implemented online science course content materials into the classes,” and 
“teachers and coaches extended the mentoring process through the semester,” and finally “teachers were better 
prepared instructors.” 

Time constraints were most often expressed as impediments to the coaching process, other impediment 
comments included: 

• “Trying to line up calendars with teachers in the program was difficult. ” 

• “Finding time to do it while working around my schedule was hard. ” 

• “Low buy-in from teachers in program. ” 

• “An in-person meeting between program coordinators, coaches and teachers to clarify expectations 

and accountability measures so that everyone is aware and on the same page. ” 

3.9.2 Implementation 

The project-designed survey item related to implementation stated, “I was able to effectively implement some of 
the course content in my class.” On a 4-point Likert scale, participants’ indicated a mean rating of 3.25 for NSCI 
650, 2.78 for NSCI 620, and 2.88 for NSCI 619. Participant responses were evenly distributed across the 4-point 
Likert scale, demonstrating a wide variation in personal experiences with implementing course content. Barriers 
to implementation were mostly related to external factors, comments related to barriers included: 

• “Tightly circumscribed Colorado Content Standards impaired the implementation of specific content in 
my classroom, but did not impair my own learning. ” (Fall 2011) 

• “My current district is very conservative with no interest in progressive education. Hands-on activities 
are discouraged as well as teaching any form of controversy. My administrator is a barrier, but I think 
I’m learning wonderful things for my next job. ” (Spring 2011) 

• “Implementation was very difficult because they (course topics) did not fit with the current unit I was 
teaching and were so far beyond what I would be teaching my students. ” (Spring 2011) 

• “How do you bring college level material down to the middle school level, others in the class did it just 
fine but I would hope that the professor could do it the best. ” (Fall 2011) 

4. Discussion 

The Improving Teacher Quality project offered an exclusive opportunity for science teachers in the Alliance 
High Schools to improve their skills and other licensed teachers to also become highly qualified to teach science. 
CSU and the supporting entities involved with project offered practicing teachers the opportunity to engage in a 
comprehensive opportunity for professional development. The grant funded project was a thoughtful and 
planned initial attempt to provide professional development to geographically distant participants. Online 
graduate level science coursework and in-class mentorship within their primary environment was combined with 
significant financial incentives to further develop the confidence and competence for the participating secondary 
science teachers. Many challenges were encountered during the course of the project that impacted the overall 
performance and success of the project, but multiple successes were achieved and many informative conclusions 
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were discovered that will guide future efforts of this project type. 

4.1 Indicators of Success 

Utilizing the information and data collected during this project, evidence of success was noted in the following 
areas: 

• Teacher Knowledge - DTAMS data in life science, earth and physical science indicated modest 
increases in content knowledge over the course of the grant period. 

• Pedagogy - DTAMS data in earth science indicated no change in scores, a modest increase in life 
sciences and a significant increase in physical science scores. Physical science was emphasized in all 
three of the online science courses participants engaged in during the grant period. 

• Positive professional effects of collaboration in online courses and in-school mentoring were reported 
by participants, instructional coaches and the school administrators. Participants indicated a very high 
rating for the statements “My coach has helped me to become a more reflective educator, ” and “I am 
satisfied with the overall support I received from my coach, ” on the final project-designed survey. 

• Noted benefits of participation were the development and enhancement of coaching and mentoring 
skills for administrators and instructional coaches that extend beyond the project and positively affect 
the school environment. 

• 88% of respondents indicated that participation in the project was viewed as positive to extremely 
positive on their professional life as a science educator. 

Statements related to the participants’ evaluation of the overall experience of the project included: 

• “Overall, this was a great benefit to my education and in turn to the education of my students...I am 
thankful for this opportunity to participate in the program and only wished that it extended the rest of 
the way through the online master program. ” 

• “I enjoyed participating in this program. I am very thankful to be a Colorado Teacher Quality Grant 
awardee. The graduate courses helped increase my knowledge of current scientific issues, as well as 
enrich my classroom with some new, innovative labs. I recommend this program to others who are 
interested in increasing their content knowledge while continuing to teach. ” 

• “I am a much stronger science teacher because of the Colorado Teacher Quality Grant! Because I have 
learned so much, I have much more to offer my students... Thank you so much!’’ 

4.2 Challenges to Success 

The challenges to success were indicated in multiple areas of the project, and most challenges may be 
attributable to the exploratory nature of the project and the accelerated timeline between grant proposal 
submission, acceptance of grant and the initiation of the program. The first challenge the project encountered 
was the University professors’ inexperience with teaching graduate level courses utilizing an online format. The 
professors’ had to manage organization and presentation of content and integrate student participation into 
courses only taught in person previously. In addition, professors needed to adapt the science course material to 
be orientated to educators. Integration of basic pedagogical concepts, methods for adapting material to various 
developmental levels, use of a lesson plan format, and methods for assessment were absent during the initial 
course. A participant commented, “I wish there would have been more thought put into the classes and how they 
could help teachers improve students assessments. ” Utilizing participant feedback, subsequent courses attempted 
to make efforts to integrate pedagogical methods and concepts to provide a more relevant course for educators. 
The second challenge of the project involved the pedagogical support of participants with an instructional coach. 
Due to the constraint of time for the initiation of the project, identification of appropriate coaches was 
challenging. As a result, the selected coaches were chosen out of convenience, rather than selected on 
pedagogical content knowledge in science education. Coaching emphasis was thus placed on the mechanics of 
teaching practice and neglected the meaningful integration of science concepts and principles into class 
instruction. The final challenge indicated was related to the science content presented in the three courses. 
University instructors emphasized that the creation of a theme of interest to connect science concepts would be 
helpful to participants. Due to school, district, and state science standards, participants found the single theme to 
be too restrictive and unrelated to the topics they were required to teach. 

4.3 Recommendations for Success 

Participants expressed an appreciation of the hands-on labs as contributors to their understanding of the course 
content and its potential use in classroom instruction. Utilization of a more thoughtful integration of hands-on lab 
exercises in online courses to enhance participant learning and provide methods for classroom integration. 
Additionally, a more comprehensive and focused assessment methodology related to the lab portion of the 
course may be beneficial to instructor and participant. 


49 




Journal of Education and Practice 

ISSN 2222-1735 (Paper) ISSN 2222-288X (Online) 

Vol.6, No.3, 2015 


www.iiste.org 


JILU 

i$te 


Lesson plan tasks were found to be difficult to integrate in course requirements due to individual 
schools’ expectations and varying formats utilized. Issues related to completion of the lesson plan requirement 
for courses may have compromised the effectiveness of the assignment. Use of the lesson plan, its design and 
implementation may be most effective if collaboration on the specific format includes the contribution of the 
participating entities and the specific expectations. 

The connection or lack thereof between the course content and classroom pedagogy was a significant 
challenge identified throughout the project. Perhaps, one of the major contributing factors to the attrition of 
participants. Participants engaged in the program to develop and enhance their ability to educate students in 
secondary science, a reasonable measure of success would be their confidence and competence at the conclusion 
of the program. Comments related to this issue stated in survey data included, “I would like to have a master 
teacher (who has taught science to 6 th - 12 th graders) provide feedback on the lesson plans,” and “I would 
suggest having an education instructor involved in the course in some way as the support on implementation of 
the content into the classroom was one area that I felt was severely lacking .” In order to address and align the 
graduate courses to meet the pedagogical connection, many educational elements would need to be integrated. 
Elements may include content standards for the intended audience, lesson plan requirements, pre/post 
assessments, grading rubrics for assignments, additional lab activities. Finally, alignment of consecutive courses 
that maintain a similar features and limit the adjustment of participants to new procedures. 

The use of technologies for course delivery was a critical element for this project, in that the 
participants were geographically scattered around the state. The inclusion of digital resources and technology 
was rated high on survey data. Due to the nature of the method of content learning, continued exploration to 
enhance the informational and technological experience for participants is needed. Utilization of web based 
applications, interactive and educational websites, software, simulations, video, and other related media should 
be included. 

Engagement of teachers in professional learning communities related to course content was an element 
that was increasingly utilized throughout the program. Course instructors and participants contributed to online 
discussions and peer communication activities enhanced the learning experience. Course designers should 
continue to explore new and exciting ways of encouraging this collaboration within and outside the course. 
Further development of communication techniques to foster open communication among students and instructors 
will serve to continue to develop an understanding of the concepts and principles. Models of STEM related 
content should be explored to determine the most effective and relevant mode of professional development 
content for current educators. Finally, future research should continue to explore the most meaningful and 
effective delivery for providing current educators with updated STEM content. 
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